Research Methods 2022

Ethan Milne // Ivey Business School

11/30/22

Gameplan:


Why webscrape?


Scraping with APIs


Scraping without APIs

Why Webscrape?

What is Webscraping?

Why Webscrape?

Why Webscrape?


New Phenomena


Ecological Value


Methods Advancement


Improving Measurement

The Webscraping Iceberg

Scraping with APIs

What is an API?


Application Program Interface

APIs: Pros and Cons


Pros

Cons

Easy to use Not available for every site
Well-documented No control over data available
Limited liability Usage restrictions

Brady (2021)


1. So what?

Brady (2021)


1. So what?


2. Why scrape?

Twitter Study


library(academictwitteR)

bk_tweets <- get_all_tweets(
  query = "@BurgerKingUK",
  start_tweets = "2021-03-06T00:00:00Z",
  end_tweets = "2021-03-13T00:00:00Z",
  n = 10000000
)


library(academictwitteR)

bk_tweets <- get_all_tweets(
  query = "@BurgerKingUK",
  start_tweets = "2021-03-06T00:00:00Z",
  end_tweets = "2021-03-13T00:00:00Z",
  n = 10000000
)

for (i in unique_authors) {
  past_tweets[[i]] <- get_all_tweets(
    users = unique_authors$username[start:end],
    start_tweets = "2021-03-01T00:00:01.000Z",
    end_tweets = "2021-03-08T00:00:01.000Z",
    n = 1000000
  )
}


Results

Model Summaries
Characteristic Without Present Tweets With Present Tweets
IRR1 95% CI1 p-value IRR1 95% CI1 p-value
NormalQuantity (past) 0.70 0.68, 0.73 <0.001 0.74 0.71, 0.76 <0.001
NormalLikes (past) 0.96 0.93, 0.98 <0.001 0.93 0.91, 0.96 <0.001
OutrageQuantity (past) 1.72 1.64, 1.81 <0.001 1.68 1.61, 1.76 <0.001
OutrageLikes (past) 1.12 1.09, 1.15 <0.001 1.09 1.06, 1.13 <0.001
Followers 1.00 1.00, 1.00 0.4 1.00 1.00, 1.00 >0.9
TotalTweets 1.00 1.00, 1.00 <0.001 1.00 1.00, 1.00 <0.001
NormalQuantity (present) 4.08 3.95, 4.21 <0.001
1 IRR = Incidence Rate Ratio, CI = Confidence Interval

Scraping Without APIs

Building scrapers is hard


No prebuilt code


Websites not designed for scraping


Websites protected against scraping

Building scrapers is rewarding


You know your data inside and out


Your data is unique


Your scraper is a contribution

Structured: Wikipedia


library(htmltab)


library(htmltab)

htmltab("https://en.wikipedia.org/wiki/List_of_Game_of_Thrones_episodes", 
        2) |>
  select(c(1,2,7))
   No.overall No. inseason U.S. viewers(millions)
2           1            1                   2.22
3           2            2                   2.20
4           3            3                   2.44
5           4            4                   2.45
6           5            5                   2.58
7           6            6                   2.44
8           7            7                   2.40
9           8            8                   2.72
10          9            9                   2.66
11         10           10                   3.04

Semi-Structured: Fanfiction

Fanfiction


Why Scrape?


Get HTML pages


library(httr)
library(rvest)

get_html <- function(input) {
  GET(
    input, #insert URL to scrape
    user_agent(
      "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36"
    )
  ) |>
    read_html()
}

Get HTML


get_html(input = "https://archiveofourown.org/tags/Fantastic%20Beasts%20and%20Where%20to%20Find%20Them%20(Movies)/works")
{html_document}
<html lang="en">
[1] <head>\n<meta http-equiv="Content-Type" content="text/html; charset=UTF-8 ...
[2] <body class="logged-out">\n    <div id="outer" class="wrapper">\n      <u ...

Read HTML


get_html(input = "https://archiveofourown.org/tags/Fantastic%20Beasts%20and%20Where%20to%20Find%20Them%20(Movies)/works") |>
  html_text2()
[1] "Main Content\n\nWhile we've done our best to make the core functionality of this site accessible without javascript, it will work better with it enabled. Please consider turning it on!\n\nArchive of Our Own beta\n\nLog In\n\nUser name or email:\nPassword:\n\nRemember Me\n\nForgot password?\nGet an Invitation\nSite Navigation\nFandoms\nAll Fandoms\nAnime & Manga\nBooks & Literature\nCartoons & Comics & Graphic Novels\nCelebrities & Real People\nMovies\nMusic & Bands\nOther Media\nTheater\nTV Shows\nVideo Games\nUncategorized Fandoms\nBrowse\nWorks\nBookmarks\nTags\nCollections\nSearch\nWorks\nBookmarks\nTags\nPeople\nAbout\nAbout Us\nNews\nFAQ\nWrangling Guidelines\nDonate or Volunteer\nSearch Works\n\nWork Search: tip: lex m/m (mature OR explicit)\n\n1 - 20 of 15729 Works in Fantastic Beasts and Where to Find Them (Movies)\nNavigation and Actions\nWorks\nBookmarks\nFilters\nRSS Feed\nPages Navigation\n← Previous\n1\n2\n3\n4\n5\n6\n7\n8\n9\n…\n786\n787\nNext →\nListing Works\nMagical Marvel (HP X MCU) SI by do not cache athass_prkr\nFandoms: Harry Potter - J. K. Rowling, Marvel Cinematic Universe, The Avengers (Marvel Movies), The Avengers (Marvel) - All Media Types, X-Men - All Media Types, Fantastic Beasts and Where to Find Them (Movies), Fantastic Four, Spider-Man - All Media Types \nMature\nChoose Not To Use Archive Warnings\nF/F, F/M, Gen, Multi\nWork in Progress\n\n06 Oct 2022\n\nTags\nCreator Chose Not To Use Archive Warnings\nJames Potter/Lily Evans Potter\nJasmine Potter (OFC)\nLily Evans Potter\nJames Potter\nAlbus Dumbledore\nRose Potter (OFC)\nNathan Potter (OMC)\nSteve Rogers\nAncient One (Marvel)\nMerlin (Harry Potter)\nStephen Strange\nGodric Gryffindor\nSalazar Slytherin\nRowena Ravenclaw\nHelga Hufflepuff\nMorgana | Morgan le Fay (Harry Potter)\nMordred (Arthurian)\nCharles Xavier\nEn Sabah Nur\nErik Lehnsherr\nHela (Marvel)\nThor (Marvel)\nOdin (Marvel)\nTony Stark\nPeter Parker\nTom Riddle | Voldemort\nGellert Grindelwald\nNatasha Romanov (Marvel)\nWanda Maximoff\nDeath (Harry Potter)\nThe Morrigan (Marvel)\nTime Travel\nSelf-Insert\nReincarnation\nWrong Boy-Who-Lived (Harry Potter)\nAtlantis\nAncient Greece\nCamelot\nHogwarts Founders Era\nAncient Egypt\nImmortality\nMaster of Death (Harry Potter)\nJasmine in history\nLegends\nHarry Potter in MCU\nMutant Powers\nWadjet\nMedusa - Freeform\nReferences to Ancient Greek Religion & Lore\nSummary\n\n[A SI Harry Potter and Marvel (MCU) Xover Fanfiction]\n\n---------------------------------------------------------------\n\nA sick 16 year old MC dies. At the same time, emerald eyes open up confused. Shouldn't I be dead? Why the hell am I so tiny? What's her name? What are these voices in my head?\nFollow the young telepath as they make way in a world of gods and magic. Will they make their mark in this scary world or did they already do so. F*ck, time travel is really confusing.\n\n----------------------------------------------------------------\nI know that the beginning is not that good, tbh. It gets better with time, believe it.\n\nI don't own pretty much anything in this Fanfic.\n\nI took a lot of inspiration from the fic \"A Long Journey Home\" and used the prologue from \"Rise of Arcane\" to get things started.\n\n\nLanguage:\nEnglish\nWords:\n521,359\nChapters:\n231/?\nComments:\n39\nKudos:\n314\nBookmarks:\n108\nHits:\n11847\nFrom the Ashes, We Fall (Dumbledore x Reader x Grindelwald) by do not cache Monochromatically\nFandoms: Fantastic Beasts and Where to Find Them (Movies), Harry Potter - J. K. Rowling \nTeen And Up Audiences\nNo Archive Warnings Apply\nMulti\nComplete Work\n\n06 Oct 2022\n\nTags\nNo Archive Warnings Apply\nAlbus Dumbledore/Gellert Grindelwald\nAlbus Dumbledore/Gellert Grindelwald/Reader\nAlbus Dumbledore/Reader\nGellert Grindelwald/Reader\nAlbus Dumbledore\nGellert Grindelwald\nReader\nMovie 3: Fantastic Beasts: The Secrets of Dumbledore\nAlternate Universe\nCanon Rewrite\nToxic Relationships\nAlternate Universe - Canon Divergence\nPolyamory\nPolyamorous Character\nWhy Did I Write This?\nReader-Insert\nGender-neutral Reader\nno y/n\nSummary\n\nYou smile bitterly. ‘Always with the theatrics.’ you think, as you walk up to his table. You observe as he stirs his teacup, his brow quirking as it does when he is deep in thought. He barely looks any different from when you last met, with the same rugged hair and scruffy beard. Everything between you has been torn asunder, yet his familiarity still…\n\nA re-write of the beginning scene in Secrets of Dumbledore with Reader in a poly relationship with Dumbledore and Grindelwald. Reader is gender-neutral. Nothing graphic.\n\nLanguage:\nEnglish\nWords:\n1,170\nChapters:\n1/1\nKudos:\n1\nBookmarks:\n1\nHits:\n13\nIdiosyncratic Boy by do not cache uriedaysx\nFandoms: Fantastic Beasts and Where to Find Them (Movies) \nTeen And Up Audiences\nChoose Not To Use Archive Warnings\nM/M, Other\nWork in Progress\n\n06 Oct 2022\n\nTags\nCreator Chose Not To Use Archive Warnings\nNewt Scamander\nTheseus Scamander\nFluff and Angst\nLGBTQ Character\nTrans Male Character\nSummary\n\nElliot N. Scamander is the definition of Idiosyncratic. He's like his father in many, many ways, but leads a life of his own. His own identity, his own friends, loved ones, views on life, etc. His life is different from others, but it is his own, and he wouldn't have it any other way.\n\nLanguage:\nEnglish\nWords:\n1,814\nChapters:\n3/?\nHits:\n10\nCrimson Destiny (Fantastic Beasts) by do not cache InGlorious_Me\nFandoms: Fantastic Beasts and Where to Find Them (Movies), Harry Potter - J. K. Rowling \nMature\nGraphic Depictions Of Violence\nF/F, Multi\nWork in Progress\n\n06 Oct 2022\n\nTags\nGraphic Depictions Of Violence\nOriginal Female Character(s)/Original Female Character(s)\nOriginal Female Human Character(s)\nOriginal Main Character (Female)\nOriginal Male Character(s)\nGellert Grindelwald\nArcturus Black III | Sirius Black's Grandfather\nSirius Black II | Hesper Gamp Black's Husband\nCygnus Black II | Violetta Bulstrode Black's Husband\nAlbus Dumbledore\nAberforth Dumbledore\nCredence Barebone\nNewt Scamander\nTheseus Scamander\nJacob Kowalski\nMinerva McGonagall\nDark Magic\nDark MC\nEvil MC\nWizarding Supremacy\nPolitics\nworld building\nKingdom building\nMagical Cores\nMagically Powerful Main Character\nLawful Evil Main Character\nHarem\nYuri-Harem\nNo-FUTA\nNo-NTR\nCruelty\nAbuse\ntorture scenes\nMinor Character Deaths\nSlow Paced\ndetailed\nOther Additional Tags to Be Added\nSummary\n\nAbandoned right after she was born, the MC was left all alone to brave the cruel world on her own. That is until a day when a man saves her from a cruel fate. See how the man known as Gellert Grindelwald becomes her guide, mentor, her father figure throughout the years.\n\nThis is the story of Destiny Grindelwald, the heir of Gellert Grindelwald. Follow her as she braves the cruel world. Watch her as she creates a better world for the Magical Population...\n\nLanguage:\nEnglish\nWords:\n29,102\nChapters:\n4/?\nKudos:\n2\nHits:\n54\nUncle Percy by do not cache Umeko\nFandoms: Fantastic Beasts and Where to Find Them (Movies) \nTeen And Up Audiences\nChoose Not To Use Archive Warnings\nF/M\nWork in Progress\n\n06 Oct 2022\n\nTags\nCreator Chose Not To Use Archive Warnings\nTina Goldstein/Newt Scamander\nTina Goldstein & Original Percival Graves\nQueenie Goldstein/Jacob Kowalski\nQueenie Goldstein & Original Percival Graves\nQueenie Goldstein & Tina Goldstein\nOriginal Percival Graves & Seraphina Picquery\nOriginal Percival Graves/Seraphina Picquery\nTina Goldstein\nQueenie Goldstein\nOriginal Percival Graves\nJacob Kowalski\nNewt Scamander\nTheseus Scamander\nSeraphina Picquery\nOriginal Characters\nMacDuff (Fantastic Beasts)\nMACUSA Staff\nGellert Grindelwald\nOriginal Percival Graves' Mother\nOriginal Percival Graves' Father\nOriginal House-Elf Character(s)\nFamily Drama\nAngst\nImplied/Referenced Torture\nMental Institutions\nMental Health Issues\nPhysical Disability\nRehabilitation\nImplied/Referenced Abuse\nDysfunctional Family\nFriendship\nLoss\nRomance\nSuicide Attempt\nPost-Traumatic Stress Disorder - PTSD\nSummary\n\nPercival Graves did not expect that accepting his mentor's dinner invitation would lead to a long-lasting friendship with two sisters.\n\nLanguage:\nEnglish\nWords:\n28,459\nChapters:\n14/?\nComments:\n29\nKudos:\n31\nBookmarks:\n1\nHits:\n724\n[GGAD]短篇合集 by do not cache A_Midwinter_Night_Dream\nFandoms: Fantastic Beasts and Where to Find Them (Movies) \nGeneral Audiences\nChoose Not To Use Archive Warnings\nM/M\nWork in Progress\n\n06 Oct 2022\n\nTags\nCreator Chose Not To Use Archive Warnings\nAlbus Dumbledore/Gellert Grindelwald\nAlbus Dumbledore\nGellert Grindelwald\nSummary\n\n在冬天的夜晚里\n\nLanguage:\n中文-普通话 國語\nWords:\n890\nChapters:\n2/?\nKudos:\n5\nHits:\n78\nleft right left right by do not cache hellhoundsprey for Lilith_the_ancient\nFandoms: Fantastic Beasts and Where to Find Them (Movies) \nExplicit\nNo Archive Warnings Apply\nM/M\nComplete Work\n\n06 Oct 2022\n\nTags\nNo Archive Warnings Apply\nOriginal Percival Graves/Theseus Scamander\nOriginal Percival Graves\nTheseus Scamander\nMilitary Backstory\nDrunk Sex\nUnresolved Romantic Tension\nPublic Sex\nHand Jobs\nSummary\n\nLet’s grab some drinks after work, he said. It’ll be fun, he said. And Theseus had indulged the old fool, of course. (Bingo square 01: uniform.)\n\nSeries\nPart 2 of Fantastic Kinks And Where To Find Them\nLanguage:\nEnglish\nWords:\n903\nChapters:\n1/1\nKudos:\n3\nHits:\n26\nWorrying Means You Suffer Twice by do not cache exasperatedtinagoldstein\nFandoms: Fantastic Beasts and Where to Find Them (Movies), Harry Potter - J. K. Rowling \nMature\nNo Archive Warnings Apply\nF/M, Gen\nWork in Progress\n\n06 Oct 2022\n\nTags\nNo Archive Warnings Apply\nTina Goldstein/Newt Scamander\nQueenie Goldstein/Jacob Kowalski\nTheseus Scamander/Eulalie Hicks\nNewt Scamander\nTina Goldstein\nNewt Scamander's Niffler\nDougal the Demiguise (Fantastic Beasts)\nPickett the Bowtruckle (Fantastic Beasts)\nOriginal Characters\nQueenie Goldstein\nJacob Kowalski\nTheseus Scamander\nEulalie Hicks\nAchilles Tolliver\nBunty (Fantastic Beasts)\nAlbus Dumbledore\nCute Kids\nProtective Tina Goldstein\nMarried Tina Goldstein/Newt Scamander\nParents Tina Goldstein/Newt Scamander\nProtective Newt Scamander\nNewt Scamander Needs a Hug\nTina goldstein needs a hug\nOriginal Percival Graves & Theseus Scamander Friendship\nScamander brothers\ngoldstein sisters\nJacob is an honorary Scamander\nLally is an honorary Goldstein\nPost-Movie 2: Fantastic Beasts: The Crimes of Grindelwald\nPost-Movie 1: Fantastic Beasts and Where to Find Them\nSummary\n\nAll Tina wants is for her husband to home, her kids to be happy and healthy, and this never ending war to be over. Newt just wants to his family to stay safe. Follow the Goldstein-Scamanders as they balance facing off against the darkest wizard in over a century and keeping the family they’ve created out of his path.\n\nSeries\nPart 16 of “Worrying Means You Suffer Twice” Universe\nLanguage:\nEnglish\nWords:\n186,272\nChapters:\n42/?\nComments:\n191\nKudos:\n239\nBookmarks:\n18\nHits:\n8316\n[GGAD/ADGG] Try and Stop Me by do not cache AsphodelV\nFandoms: Harry Potter - J. K. Rowling, Fantastic Beasts and Where to Find Them (Movies) \nMature\nChoose Not To Use Archive Warnings\nM/M\nWork in Progress\n\n05 Oct 2022\n\nTags\nCreator Chose Not To Use Archive Warnings\nAlbus Dumbledore/Gellert Grindelwald\nAlbus Dumbledore & Gellert Grindelwald\nAlbus Dumbledore\nGellert Grindelwald\nNewt Scamander\nVinda Rosier\nAberforth Dumbledore\nTime Travel\nTime Travel Fix-It\nAlternate Universe - Time Travel\nPOV Gellert Grindelwald\nSuicidal Thoughts\nSummary\n\nGellert Grindelwald travels back to the duel of 1945, and gets urges to kill himself. Hopefully, Albus can help with that.\n\nLanguage:\nEnglish\nWords:\n9,868\nChapters:\n6/?\nComments:\n33\nKudos:\n190\nBookmarks:\n17\nHits:\n2460\nThat Summer by do not cache MimmyWrites\nFandoms: Fantastic Beasts and Where to Find Them (Movies), Harry Potter - J. K. Rowling \nMature\nNo Archive Warnings Apply, Underage\nM/M\nWork in Progress\n\n05 Oct 2022\n\nTags\nNo Archive Warnings Apply\nUnderage\nAlbus Dumbledore/Gellert Grindelwald\nAlbus Dumbledore\nGellert Grindelwald\nAberforth Dumbledore\nAriana Dumbledore\nElphias Doge\nCanonical Character Death\nHomophobic Language\nViolence\nMaking Out\nBlood Pact\nSummer of 1899\nUnsympathetic Aberforth\nKinda\nSummary\n\nIn the summer of 1899, the life of two young wizards changed forever. With the arrival of Gellert Grindelwald to the Hollow, Albus Dumbledore's life took a turn for the best... ambition, darkness, and love took over the young wizard in their rawest way.\n\nSeries\nPart 4 of Grindeldore fics\nLanguage:\nEnglish\nWords:\n22,114\nChapters:\n15/20\nComments:\n18\nKudos:\n67\nBookmarks:\n10\nHits:\n2480\nPassed around Dark Lords by do not cache Vaeryn3947\nFandoms: Harry Potter - J. K. Rowling, Fantastic Beasts and Where to Find Them (Movies) \nNot Rated\nChoose Not To Use Archive Warnings\nM/M\nWork in Progress\n\n05 Oct 2022\n\nTags\nCreator Chose Not To Use Archive Warnings\nDraco Malfoy/Tom Riddle\nDraco Malfoy/Tom Riddle | Voldemort\nAlbus Dumbledore/Gellert Grindelwald\nDraco Malfoy\nTom Riddle\nAlbus Dumbledore\nGellert Grindelwald\nNewt Scamander\nTheseus Scamander\nAbraxas Malfoy\nTom Riddle | Voldemort\nSane Tom Riddle\nSane Voldemort (Harry Potter)\nDraco Malfoy Needs a Hug\nDraco Malfoy is a Brat\nImplied/Referenced Child Abuse\nAbuse\nChild Abuse\nPast Abuse\nEmotional/Psychological Abuse\nPhysical Abuse\nEmotional Manipulation\nManipulation\nMind Manipulation\nTime Travel\nTime Travelling Draco Malfoy\nAlternate Universe - Time Travel\nMovie 3: Fantastic Beasts: The Secrets of Dumbledore\nPossessive Tom Riddle\nYoung Tom Riddle\nManipulative Tom Riddle\nAlbus Dumbledore Bashing\nYoung Albus Dumbledore\nManipulative Gellert Grindelwald\nDraco Malfoy Has Issues\nDraco Malfoy Has PTSD\nDraco Malfoy Has Daddy Issues\nDraco Malfoy Has an Eating Disorder\nDraco Malfoy Has Nightmares\nTeenage Tom Riddle\nSummary\n\nDraco thought that he would have some time to think, but he didn't think he would be given several decades worth. It was just his luck to be caught in the middle of yet another fight between a Dark Lord and his little obsession. He had no idea what to even do, but when they started attacking him...well he attacked back. However, four on one to a fifteen year old that is sleep deprived, magically exhausted from however it was that he was jerked backwards in time, and still slightly traumatized from meeting the Dark Lord was definitely not good odds for him.\n\nNow he is once again caught up in Dark Lords...and why is it that they all seem to have some sort of affinity towards him for whatever reason, and also Dumbledore...what is the reason for that? First Voldermort, and now Grindelwald. Also, why is it that they all seem to think he would be a good spy? Either way, Draco is not given many choices, it is either infiltrate Hogwards to spy on that idiot Dumbledore or torture, and Draco would very much prefer not to be tortured. Not to mention, he hates Dumbledore anyways. It is not as though he minds the task....thought...it is quite tedious really.\n\nSeries\nPart 11 of Harry Potter Stories\nPart 1 of Time Travel\nLanguage:\nEnglish\nWords:\n13,135\nChapters:\n10/?\nComments:\n23\nKudos:\n265\nBookmarks:\n37\nHits:\n4480\n1899 by do not cache AtreborAntares\nFandoms: Fantastic Beasts and Where to Find Them (Movies), Harry Potter - J. K. Rowling \nExplicit\nGraphic Depictions Of Violence\nF/M, M/M\nWork in Progress\n\n05 Oct 2022\n\nTags\nGraphic Depictions Of Violence\nAlbus Dumbledore/Gellert Grindelwald\nAlbus Dumbledore\nGellert Grindelwald\nAberforth Dumbledore\nAriana Dumbledore\nBathilda Bagshot\nCommunity: grindeldore\nAlbus Dumbledore and Gellert Grindelwald's Summer Together in 1899\nFalling In Love\nEmotional/Psychological Abuse\nEmotional Manipulation\nManipulative Relationship\nManipulative Albus Dumbledore\nManipulative Gellert Grindelwald\nYoung Albus Dumbledore/Gellert Grindelwald\nBoys In Love\nLove\nGrindelMads\nDark Magic\nDuelling\nGay Sex\nRough Sex\nSlow Burn\nDrama & Romance\nGellert Grindelwald Played by Mads Mikkelsen\nYoung Mads Mikkelsen\nAnal Sex\nAnal Fingering\nCharacter Study\nPsychology\nCharacter Analysis\nIn Character\nCanon\nSummary\n\nGodric’s Hollow, 1899.\nLa mia personale versione del 1899, la storia d’amore tra Albus Silente e Gellert Grindelwald, momento dopo momento.\nCiò che J. K. Rowling non ci ha raccontato e che siamo liberi d’immaginare.\n\nLanguage:\nItaliano\nWords:\n71,867\nChapters:\n16/?\nComments:\n63\nKudos:\n63\nBookmarks:\n3\nHits:\n1310\nCountryside by do not cache StarTalers\nFandoms: Fantastic Beasts and Where to Find Them (Movies), Harry Potter - J. K. Rowling \nTeen And Up Audiences\nNo Archive Warnings Apply\nM/M\nWork in Progress\n\n05 Oct 2022\n\nTags\nNo Archive Warnings Apply\nAlbus Dumbledore/Gellert Grindelwald\nGellert Grindelwald\nAlbus Dumbledore\nSummary\n\n*Original:《Fantastic Beats: The Crimes of Grindewald》[Warner Bros. Pictures]\n*CP:格林德沃×邓布利多\n*Note:年龄操作,老盖小邓|整点地摊文学封建余孽ABO!!\n\nLanguage:\n中文-普通话 國語\nWords:\n205,812\nChapters:\n40/?\nCollections:\n1\nComments:\n71\nKudos:\n297\nBookmarks:\n39\nHits:\n14405\n【Thesewt】人无完人 by do not cache FiveSevenNine\nFandoms: Fantastic Beasts and Where to Find Them (Movies) \nTeen And Up Audiences\nChoose Not To Use Archive Warnings\nNo category\nComplete Work\n\n05 Oct 2022\n\nTags\nCreator Chose Not To Use Archive Warnings\nthesewt\nNewt Scamander & Theseus Scamander\nTop Theseus Scamander\nBottom Newt Scamander\n忒纽\nSummary\n\n纽特不是厨房杀手,他只是,嗯,不擅长做熟食\n\nLanguage:\n中文-普通话 國語\nWords:\n1,568\nChapters:\n1/1\nKudos:\n5\nHits:\n35\nMACUSA's Resident Ghost by do not cache Umeko\nFandoms: Fantastic Beasts and Where to Find Them (Movies) \nTeen And Up Audiences\nMajor Character Death\nGen\nComplete Work\n\n05 Oct 2022\n\nTags\nMajor Character Death\nOriginal Percival Graves\nGellert Grindelwald\nSeraphina Picquery\nTina Goldstein\nQueenie Goldstein\nGhosts\nMACUSA | Magical Congress of the United States of America\nHumor\nSummary\n\nAn early Halloween fic. An unwitting ghost ends up stuck in his MACUSA office, literally.\n\nLanguage:\nEnglish\nWords:\n1,529\nChapters:\n1/1\nComments:\n2\nKudos:\n2\nHits:\n26\nkilled the cat by do not cache hellhoundsprey for Lilith_the_ancient\nFandoms: Fantastic Beasts and Where to Find Them (Movies) \nMature\nNo Archive Warnings Apply\nM/M\nComplete Work\n\n04 Oct 2022\n\nTags\nNo Archive Warnings Apply\nCredence Barebone/Original Percival Graves\nCredence Barebone\nOriginal Percival Graves\nPining\nPorn Watching\nTouch-Starved\nImplied/Referenced Alcohol Abuse/Alcoholism\nSummary\n\nCredence’s curiosity gets the better of him. (Bingo square 04: pictures.)\n\nSeries\nPart 1 of Fantastic Kinks And Where To Find Them\nLanguage:\nEnglish\nWords:\n1,790\nChapters:\n1/1\nComments:\n2\nKudos:\n6\nHits:\n51\nRegretfully Uncaring by do not cache RMWB\nFandoms: Harry Potter - J. K. Rowling, Fantastic Beasts and Where to Find Them (Movies) \nMature\nMajor Character Death, Graphic Depictions Of Violence, Rape/Non-Con\nF/M, M/M\nWork in Progress\n\n04 Oct 2022\n\nTags\nMajor Character Death\nGraphic Depictions Of Violence\nRape/Non-Con\nHarry Potter/Ginny Weasley\nDaphne Greengrass/Ron Weasley\nRemus Lupin/Nymphadora Tonks (minor)\nAudrey Weasley/Percy Weasley\nCharlie Weasley/Original Female Character(s) (Minor)\nSirius Black\nGinny Weasley\nHarry Potter\nRemus Lupin\nOriginal Squib Characters (Harry Potter)\nDaphne Greengrass\nRon Weasley\nFenrir Greyback\nNymphadora Tonks\nBartemius Crouch Jr.\nTom Riddle | Voldemort\nNarcissa Black Malfoy\nCharlie Weasley\nPercy Weasley\nDraco Malfoy\nColin Creevey\nGellert Grindelwald\nNewt Scamander\nTina Goldstein\nHermione Granger\nBill Weasley\nOriginal Werewolf Character(s)\nNeville Longbottom\nFleur Delacour\nDudley Dursley\nRomance\nSlow Burn\nRon Weasley Kicks Some Ass\nGinny Weasley Understands Tom Riddle\nDaphne Greengrass isn't an Ice Queen\nAlternate Universe\nHarry Potter is a werewolf\nIt's Not All About Werewolf Bullshit. At. All.\nAngst\nFeels\nTime Travel\nHorcruxes Do Cool Shit Every Time\nOriginal Plot\nMaster of Death\nSo Many Villains - Freeform\nAggressive Canon Divergence\nEnsemble Cast With Multiple POV's\nplot heavy\nDeathly Hallows\nPeverell Shinanigans\nGinny Weasley Gets Some Depth\nCanon typical humour\nNo Bashing\nFlaw Acknowledgement\nSummary\n\nHarry Potter becomes a werewolf at the end of his third year. Villains, some new, and some old make their presence known. The events that follow create an entirely different war. One in which Harry grapples with his new reality, Ginny harbours a terrible secret, Daphne has a rare power that none can explain, and Ron is stuck in the middle of it all.\n\nA werewolf AU that, quite honestly, isn’t focused on the werewolves.\n\nWriting quality, plot, and characterization improve dramatically from chapters 1-11 (Which are currently being rewritten). Read tags.\n\nLanguage:\nEnglish\nWords:\n354,850\nChapters:\n35/86\nComments:\n60\nKudos:\n97\nBookmarks:\n31\nHits:\n5675\nShaped By Adversity by do not cache draconickitty\nFandoms: Fantastic Beasts and Where to Find Them (Movies), Geneforge \nTeen And Up Audiences\nNo Archive Warnings Apply\nGen\nWork in Progress\n\n04 Oct 2022\n\nTags\nNo Archive Warnings Apply\nNewt Scamander\nAlbus Dumbledore\nAberforth Dumbledore\nGellert Grindelwald\nNewt Scamander's Mother (mentioned)\nNewt Scamander's Father (Mentioned)\nTheseus Scamander (mentioned)\nMagda (Geneforge)\nTyrol (Geneforge)\nGalatea Merrythought\nOriginal Characters\nLeta Lestrange\nPickett the Bowtruckle (Fantastic Beasts)\nNewt Scamander's Magical Beasts\nOther characters to be added as necessary - Character\nAriana Dumbledore (Mentioned)\nPast Child Abuse\nDepictions of slavery\nPresent Tense\nAddiction\nImplied/Referenced Animal Abuse\nSummary\n\nThe Shapers have the power to create life. They can make plants and animals to populate barren lands and intelligent, loyal serviles to do their menial work. And, if challenged, they can create armies of hideous monsters to devastate their foes.\n\nThe Shapers rule these lands, and their authority is absolute. There is one punishment for those who try to disobey them: death.\n\nAnd so the rebellion began.\n\nNewt Fidelis has already lost much to the Shapers. Rather than stay uninvolved, he has chosen to take the fight to them by whatever means necessary.\n\nIf only the means necessary hadn't turned out to be both volatile and uncontrollable...\n\nLanguage:\nEnglish\nWords:\n5,552\nChapters:\n2/?\nComments:\n1\nKudos:\n1\nBookmarks:\n1\nHits:\n27\n【GGAD】英雄的噩梦 by do not cache Untilyoucallmyname\nFandoms: Harry Potter - J. K. Rowling, Fantastic Beasts and Where to Find Them (Movies) \nNot Rated\nChoose Not To Use Archive Warnings\nM/M\nComplete Work\n\n04 Oct 2022\n\nTags\nCreator Chose Not To Use Archive Warnings\nAlbus Dumbledore/Gellert Grindelwald\nAlbus Dumbledore\nGellert Grindelwald\nSummary\n\n打败格林德沃之后…………一直到他死,他都在重复地做着同一个梦。\n\nLanguage:\n中文-普通话 國語\nWords:\n2,138\nChapters:\n1/1\nKudos:\n7\nHits:\n59\nPurebloods: Ancestors I by do not cache Grawps_Gurl\nFandoms: Harry Potter and the Cursed Child - Thorne & Rowling, Harry Potter - J. K. Rowling, Fantastic Beasts and Where to Find Them (Movies) \nMature\nGraphic Depictions Of Violence, Major Character Death, Underage\nF/F, F/M, M/M, Multi\nWork in Progress\n\n04 Oct 2022\n\nTags\nGraphic Depictions Of Violence\nMajor Character Death\nUnderage\nArthur Weasley/Molly Weasley\nFabian Prewett & Gideon Prewett & Molly Weasley\nBellatrix Black Lestrange/Rodolphus Lestrange\nLucius Malfoy/Narcissa Black Malfoy\nAndromeda Black Tonks/Ted Tonks\nFabian Prewett/Original Female Character(s)\nBellatrix Black Lestrange/Voldemort\nCedrella Black Weasley/Septimus Weasley\nPandora Lovegood/Xenophilius Lovegood\nAlice Longbottom/Frank Longbottom\nLucius Malfoy\nNarcissa Black Malfoy\nMarcus Flint's Father\nZabini Family (Harry Potter)\nBellatrix Black Lestrange\nAndromeda Black Tonks\nTed Tonks\nHannah Abbott's Father\nAmos Diggory\nRodolphus Lestrange\nRabastan Lestrange\nAvery Family (Harry Potter)\nEvan Rosier\nMolly Weasley\nAlice Longbottom\nFrank Longbottom\nIgnatius Prewett\nLucretia Black Prewett\nAbraxas Malfoy\nTom Riddle | Voldemort\nCantankerus Nott\nNott Sr. (Harry Potter)\nNott Family (Harry Potter)\nAngst\nChildhood Trauma\nPower Play\nFolklore\nDark Magic\nCanon Timeline\nHogwarts\nPureblood Culture (Harry Potter)\nPureblood Society (Harry Potter)\nDeath Eaters\nSquib Rights (Harry Potter)\nSummary\n\nIt would always come down to blood, wouldn’t it?\n\nLucius Malfoy, The Black Sisters, Lestrange Brothers, Prewetts, Carrows, and many more families are starting and completing their journey at Hogwarts; meanwhile, a new wizard is shaking up the wizarding world; or rather tearing it up piece by piece. Will Love, War, and Blood — lots and lots of blood — bring them down or drag them up? And whose waiting at either end? Is there anyone waiting to save them, to begin with?\n\nWhat will come of a group of kids who are forced to grow up too quickly, too isolated, and too buried in the mistakes of their own ancestors to even breath?\n\nThey always thought their bloodlines were what would save them, but their heritage can only protect them from so much, and that’s the thing with blood... you only have so much.\n\n*can live within a cannon timeline\n\nLanguage:\nEnglish\nWords:\n97,844\nChapters:\n18/?\nKudos:\n15\nBookmarks:\n2\nHits:\n286\nFilters\nFilter results:\nSubmit\nSort by\nAuthorTitleDate PostedDate UpdatedWord CountHitsKudosCommentsBookmarks\nInclude\n?\nInclude Ratings\nGeneral Audiences (4623)\nTeen And Up Audiences (4158)\nExplicit (2722)\nMature (2566)\nNot Rated (1660)\nInclude Warnings\nNo Archive Warnings Apply (7378)\nCreator Chose Not To Use Archive Warnings (6631)\nGraphic Depictions Of Violence (1099)\nRape/Non-Con (647)\nMajor Character Death (642)\nUnderage (474)\nInclude Categories\nM/M (9501)\nF/M (3959)\nGen (2542)\nMulti (565)\nF/F (419)\nOther (407)\nInclude Fandoms\nFantastic Beasts and Where to Find Them (Movies) (15729)\nHarry Potter - J. K. Rowling (3996)\nFantastic Beasts and Where to Find Them (Movies) RPF (70)\nMarvel Cinematic Universe (67)\nHarry Potter and the Cursed Child - Thorne & Rowling (54)\nThe Avengers (Marvel Movies) (39)\nBritish Actor RPF (38)\nElla Enchanted (2004) (38)\nSherlock (TV) (32)\nFright Night (2011) (28)\nInclude Characters\nNewt Scamander (8037)\nOriginal Percival Graves (5070)\nTina Goldstein (4488)\nGellert Grindelwald (4075)\nCredence Barebone (3681)\nAlbus Dumbledore (3548)\nTheseus Scamander (3343)\nQueenie Goldstein (3082)\nJacob Kowalski (2132)\nSeraphina Picquery (1125)\nInclude Relationships\nAlbus Dumbledore/Gellert Grindelwald (2814)\nTina Goldstein/Newt Scamander (2152)\nOriginal Percival Graves/Newt Scamander (1953)\nCredence Barebone/Original Percival Graves (1753)\nQueenie Goldstein/Jacob Kowalski (1096)\nNewt Scamander/Theseus Scamander (1048)\nNewt Scamander & Theseus Scamander (736)\nTina Goldstein & Newt Scamander (639)\nCredence Barebone/Newt Scamander (501)\nCredence Barebone/Percival Graves | Gellert Grindelwald (481)\nInclude Additional Tags\nFluff (2195)\nAngst (1408)\nHurt/Comfort (1152)\nAlternate Universe - Canon Divergence (917)\nRomance (890)\nAlternate Universe (759)\nAlternate Universe - Modern Setting (540)\nDrama (527)\nFluff and Angst (525)\nPlot What Plot/Porn Without Plot (508)\nOther tags to include\nExclude\n?\nExclude Ratings\nGeneral Audiences (4623)\nTeen And Up Audiences (4158)\nExplicit (2722)\nMature (2566)\nNot Rated (1660)\nExclude Warnings\nNo Archive Warnings Apply (7378)\nCreator Chose Not To Use Archive Warnings (6631)\nGraphic Depictions Of Violence (1099)\nRape/Non-Con (647)\nMajor Character Death (642)\nUnderage (474)\nExclude Categories\nM/M (9501)\nF/M (3959)\nGen (2542)\nMulti (565)\nF/F (419)\nOther (407)\nExclude Fandoms\nFantastic Beasts and Where to Find Them (Movies) (15729)\nHarry Potter - J. K. Rowling (3996)\nFantastic Beasts and Where to Find Them (Movies) RPF (70)\nMarvel Cinematic Universe (67)\nHarry Potter and the Cursed Child - Thorne & Rowling (54)\nThe Avengers (Marvel Movies) (39)\nBritish Actor RPF (38)\nElla Enchanted (2004) (38)\nSherlock (TV) (32)\nFright Night (2011) (28)\nExclude Characters\nNewt Scamander (8037)\nOriginal Percival Graves (5070)\nTina Goldstein (4488)\nGellert Grindelwald (4075)\nCredence Barebone (3681)\nAlbus Dumbledore (3548)\nTheseus Scamander (3343)\nQueenie Goldstein (3082)\nJacob Kowalski (2132)\nSeraphina Picquery (1125)\nExclude Relationships\nAlbus Dumbledore/Gellert Grindelwald (2814)\nTina Goldstein/Newt Scamander (2152)\nOriginal Percival Graves/Newt Scamander (1953)\nCredence Barebone/Original Percival Graves (1753)\nQueenie Goldstein/Jacob Kowalski (1096)\nNewt Scamander/Theseus Scamander (1048)\nNewt Scamander & Theseus Scamander (736)\nTina Goldstein & Newt Scamander (639)\nCredence Barebone/Newt Scamander (501)\nCredence Barebone/Percival Graves | Gellert Grindelwald (481)\nExclude Additional Tags\nFluff (2195)\nAngst (1408)\nHurt/Comfort (1152)\nAlternate Universe - Canon Divergence (917)\nRomance (890)\nAlternate Universe (759)\nAlternate Universe - Modern Setting (540)\nDrama (527)\nFluff and Angst (525)\nPlot What Plot/Porn Without Plot (508)\nOther tags to exclude\nMore Options\nCrossovers\nInclude crossovers\nExclude crossovers\nShow only crossovers\nCompletion Status\nAll works\nComplete works only\nWorks in progress only\nWord Count\nFrom\nTo\nDate Updated\nFrom\nTo\nSearch within results ?\nLanguage\naf SoomaaliAfrikaansالعربية𓂋𓏺𓈖 𓆎𓅓𓏏𓊖ܐܪܡܝܐ | ארמיאհայերենasturianuBahasa IndonesiaBahasa MalaysiaБългарскиবাংলাBasa JawaБашҡорт телебеларускаяBosanskiBrezhonegCatalàCebuanoČeštinaChinuk WawaCymraegDanskDeutscheesti keelΕλληνικά𒅴𒂠EnglishEald EnglisċEspañolEsperantoEuskaraفارسیFilipinoFrançaisFurlanGaeilgeGàidhligGalego𐌲𐌿𐍄𐌹𐍃𐌺𐌰中文-客家话한국어Hausa | هَرْشَن هَوْسَहिन्दीHrvatskiʻŌlelo HawaiʻiInterlinguaisiZuluÍslenskaItalianoעבריתಕನ್ನಡქართულიភាសាខ្មែរKhuzdulKiswahilikreyòl ayisyenKurdî | کوردیКыргызчаLangue des signes québécoiseLatviešu valodaLëtzebuergeschLietuvių kalbaLingua latinaMagyarмакедонскиമലയാളംMaltiᠮᠠᠨᠵᡠ ᡤᡳᠰᡠᠨमराठीMikisúkîᠮᠣᠩᠭᠣᠯ ᠪᠢᠴᠢᠭ᠌ | Монгол Кирилл үсэгမြန်မာဘာသာNāhuatl中文-闽南话 臺語Nederlands日本語NorskАзәрбајҹан дили | آذربایجان دیلیНохчийн моттلسان عثمانىپښتوPlattdüütschPolskiPortuguês brasileiroPortuguês europeuਪੰਜਾਬੀqazaqşa | қазақшаQuenyaRomânăРусскийScotsShqipSindarinසිංහලSlovenčinaSlovenščinaSprēkō ÞiudiskōСрпскиSuomiSvenskaதமிழ்తెలుగుไทยThermianབོད་སྐད་Tiếng Việtϯⲙⲉⲧⲣⲉⲙⲛ̀ⲭⲏⲙⲓtlhIngan-Holtoki ponaτσακώνικαTürkçeУкраїнськаاُردُوئۇيغۇر تىلىVolapük中文-吴语יידישmaayaʼ tʼàan中文-广东话 粵語中文-普通话 國語\nSubmit\n\nClear Filters\n\nTop of Work Index\n\nPages Navigation\n← Previous\n1\n2\n3\n4\n5\n6\n7\n8\n9\n…\n786\n787\nNext →\npresentational\nFooter\nAbout the Archive\nSite Map\nDiversity Statement\nTerms of Service\nDMCA Policy\nContact Us\nPolicy Questions & Abuse Reports\nTechnical Support & Feedback\nDevelopment\notwarchive v0.9.325.8\nKnown Issues\nGPL by the OTW\nif(typeof jQuery=='undefined'){document.write(unescape(\"%3Cscript src='/javascripts/jquery.min.js' type='text/javascript'%3E%3C/script%3E\"));document.write(unescape(\"%3Cscript src='/javascripts/jquery-ui.min.js' type='text/javascript'%3E%3C/script%3E\"));}$j=jQuery.noConflict();eval(mod_pagespeed_a8hDjzT3l_);eval(mod_pagespeed_1nA1vKz9z4);eval(mod_pagespeed_AKC4FLip6q);eval(mod_pagespeed_gX7XC77lvT);eval(mod_pagespeed_5zo0ZWMy4O);eval(mod_pagespeed_eCCgOhjhYQ);eval(mod_pagespeed_TU1Ln7LKSh);eval(mod_pagespeed_qTpbjsXQwh);eval(mod_pagespeed_R5FGw2ySDb);eval(mod_pagespeed_rqqFCT3xcT);eval(mod_pagespeed_Gf_4KFhTvB);function isSupported(){var item=\"localStoragePolyfill\";try{localStorage.setItem(item,item);localStorage.removeItem(item);return true;}catch(e){return false;}}function acceptTOS(){if(isSupported()){localStorage.setItem(\"accepted_tos\",\"20180523\");}else{Cookies.set(\"accepted_tos\",\"20180523\",{expires:365});}}$j(document).ready(function(){if(localStorage.getItem(\"accepted_tos\")!==\"20180523\"&&Cookies.get(\"accepted_tos\")!==\"20180523\"){$j(\"body\").prepend(\"<div id=\\\"tos_prompt\\\" class=\\\"hidden\\\">\\n <h2 class=\\\"heading\\\">\\n <span>Archive of Our Own<\\/span>\\n <\\/h2>\\n <div class=\\\"agreement\\\">\\n <p class=\\\"summary\\\">\\n On Archive of Our Own (AO3), users can make profiles, create works and\\n other Content, post comments, give Kudos, create Collections and \\n Bookmarks, participate in Challenges, import works, and more. Any \\n information you publish in a comment, profile, work, or Content that you \\n post or import onto AO3 including in summaries, notes and tags, \\n will be accessible by the public (unless you limit access to a work only \\n to those with AO3 Accounts), and it will be available to \\n AO3 personnel. Be mindful when sharing personal information, \\n including your religious or political views, health, racial background, \\n country of origin, sexual identity and/or personal relationships. To \\n learn more, check out our <a href=\\\"/tos\\\">Terms of Service<\\/a> and <a href=\\\"/tos#privacy\\\">Privacy Policy<\\/a>.\\n <\\/p>\\n\\n <p class=\\\"confirmation\\\">\\n <input type=\\\"checkbox\\\" id=\\\"tos_agree\\\" />\\n <label for=\\\"tos_agree\\\">I have read &amp; understood the new Terms of Service and Privacy Policy<\\/label>\\n <\\/p>\\n\\n <p class=\\\"submit\\\">\\n <button name=\\\"button\\\" type=\\\"button\\\" disabled=\\\"disabled\\\" id=\\\"accept_tos\\\">I agree/consent to its terms<\\/button>\\n <\\/p>\\n\\n <\\/div>\\n<\\/div>\\n\\n<script>\\n//<![CDATA[\\n\\n \\$j(document).ready(function() {\\n var container = \\$j(\\\"#tos_prompt\\\");\\n var outer = \\$j(\\\"#outer\\\");\\n var button = \\$j(\\\"#accept_tos\\\");\\n\\n setTimeout(showTOSPrompt, 1500);\\n\\n function showTOSPrompt() {\\n \\$j.when(container.fadeIn(500)).done(function() {\\n outer.addClass(\\\"hidden\\\").attr(\\\"aria-hidden\\\", \\\"true\\\");\\n });\\n\\n \\$j(\\\"#tos_agree\\\").on(\\\"click\\\", function() {\\n button.attr(\\\"disabled\\\", !this.checked);\\n if (this.checked) {\\n button.on(\\\"click\\\", function() {\\n acceptTOS();\\n outer.removeClass(\\\"hidden\\\").removeAttr(\\\"aria-hidden\\\");\\n \\$j.when(container.fadeOut(500)).done(function() {\\n container.remove();\\n });\\n });\\n };\\n }).change();\\n };\\n });\\n\\n//]]]]><![CDATA[>\\n<\\/script>\");}});$j(document).ready(function(){var permitted_hosts=[\"104.153.64.122\",\"208.85.241.152\",\"208.85.241.157\",\"archiveofourown.org\",\"download.archiveofourown.org\",\"insecure.archiveofourown.org\",\"secure.archiveofourown.org\",\"www.archiveofourown.com\",\"www.archiveofourown.net\",\"www.archiveofourown.org\",\"insecure-test.archiveofourown.org\",\"test.archiveofourown.org\",\"testdownload.archiveofourown.org\"];var current_host=window.location.hostname;if(!permitted_hosts.includes(current_host)&&Cookies.get(\"proxy_notice\")!==\"0\"&&window.location.protocol!==\"file:\"){$j(\"#skiplinks\").after(\"<div id=\\\"proxy-notice\\\">\\n <div class=\\\"userstuff\\\">\\n <p class=\\\"important\\\">Important message:<\\/p>\\n <ol>\\n <li>You are using a proxy site that is not part of the Archive of Our Own.<\\/li>\\n <li>The entity that set up the proxy site can see what you submit, including your IP address. If you log in through the proxy site, it can see your password.<\\/li>\\n <\\/ol>\\n <p class=\\\"important\\\">重要提示:<\\/p>\\n <ol>\\n <li>您使用的是第三方开发的反向代理网站,此网站并非Archive of Our Own - AO3(AO3作品库)原站。<\\/li>\\n <li>代理网站的开发者能够获取您上传至该站点的全部内容,包括您的ip地址。如您通过代理登录AO3,对方将获得您的密码。<\\/li>\\n <\\/ol>\\n <p class=\\\"submit\\\"><button class=\\\"action\\\" type=\\\"button\\\" id=\\\"proxy-notice-dismiss\\\">Dismiss Notice<\\/button><\\/p>\\n <\\/div>\\n<\\/div>\\n\\n<script>\\n//<![CDATA[\\n\\n \\$j(document).ready(function() {\\n \\$j(\\\"#proxy-notice-dismiss\\\").on(\\\"click\\\", function() {\\n Cookies.set(\\\"proxy_notice\\\", \\\"0\\\");\\n \\$j(\\\"#proxy-notice\\\").slideUp();\\n });\\n });\\n\\n//]]]]><![CDATA[>\\n<\\/script>\");}});$j(\".datepicker\").datepicker({dateFormat:\"yy-mm-dd\"});"

Build Function


get_pageworkIDs <- function(input){
  input |>
    html_elements(css = "ol.work.index.group") |>
    as.character() |>
    str_match_all("<li id=(.*?) class=") |>
    data.frame() |>
    select("X2") |>
    mutate(X2 = gsub('[[:punct:]]', "", X2),
           X2 = gsub("work", "", X2)) |>
    rename(id = X2) |>
    mutate(id = as.numeric(id),
           id = paste0("https://archiveofourown.org/works/", id))
}

Extract Information


input <- get_html(input = "https://archiveofourown.org/tags/Fantastic%20Beasts%20and%20Where%20to%20Find%20Them%20(Movies)/works")

get_pageworkIDs(input)
                                           id
1  https://archiveofourown.org/works/35525524
2  https://archiveofourown.org/works/42190833
3  https://archiveofourown.org/works/42189672
4  https://archiveofourown.org/works/41930397
5  https://archiveofourown.org/works/41465799
6  https://archiveofourown.org/works/42134328
7  https://archiveofourown.org/works/42182895
8  https://archiveofourown.org/works/38103388
9  https://archiveofourown.org/works/36514978
10 https://archiveofourown.org/works/38947869
11 https://archiveofourown.org/works/39338898
12 https://archiveofourown.org/works/40082403
13 https://archiveofourown.org/works/35118067
14 https://archiveofourown.org/works/42163890
15 https://archiveofourown.org/works/42163230
16 https://archiveofourown.org/works/42159627
17 https://archiveofourown.org/works/38958546
18 https://archiveofourown.org/works/41991417
19 https://archiveofourown.org/works/42143286
20 https://archiveofourown.org/works/41371824


GetFandomIndex <- function(fandom, pages, start = 1) {
  ### Get number of pages
  indexURL <- get_indexURL(fandom)
  index <- get_html(indexURL)
  indexpages <- get_indexpages(index)
  pagecount <- min(pages, indexpages)
  index_ids <- as.data.frame(matrix(ncol = 1,
                                    nrow = 0))
  ### Scrape each page, store in dataframe
  for (i in start:pagecount) {
    pageURL <- get_indexURLpage(indexURL, i)
    page_html <- get_html(pageURL)
    id_list <- get_pageworkIDs(page_html)
    index_ids <- rbind(index_ids, id_list)
  }
  ### Output dataframe with all fanfictions
  index_ids
}

Costs and Benefits

Fanfiction model

Conclusion

Thank you!



SEthanMilne



emilne.phd@ivey.ca

SEthanMilne



Ethan Milne

References